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I. INTRODUCTION 



A. BACKGROUND 



"And a good south wind sprung up behind; 

The Albatross did fellow. 

And every day, for food or play. 

Came tc the mariner’s hollo! 

God save thee, ancient Mariner! 

From the fiends that plague thee thus! — 

Why lock’ st thou so? — With my cross-bow 
I shot the Albatross." [The Ancient Mariner, pt. i] 

The albatross around today’s program manager neck is often 
the software subcomponent of major system acquisitions. Cost 
overruns, schedule slippages, and loss of program control 
have teen the penance for those project managers who have 
failed tc provide for software with the same intensive and 
continuing management typically rendered its hardware 
counterpart. 

Software is an intangible product that defies descrip- 
tion in an engineering sense. Only a few software products 
have ever started off with clear, unambiguous, and defini- 
tive requirement specification. Schedules and costs are 
often dictated by the system acquisition milestones and 
reviews, and not necessarily associated with the phased 
software development methodologies advocated by what has 
teen termed "software engineering 1 ". Many of the specific 
problems that surround software development and acquisition 
will be discussed in detail in the next chapter. 



1 The kev objectives of software engineering are (1) a 
well-defined methodology that addresses all phases of the 
software life cycle, (2) an established set of software 
components to document ana show traceability from one devel- 
opment step to the next, and (3) a set of predictable mile- 
stones that can be reviewed as needed [Ref. 1 : p 15]. 
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In the majority cf guidance and managerial principles 
available to assist the program manager are directed at the 
hardware end. Software is the ’’new kid on the block." It is 
that part of the system that is seldom understood and often 
mismanaged during system acquisitions. Computer hardware, on 
the other hand, has undergone remarkable improvements in 
function, size, performance, and relative cost. Several 
hardware generations have emerged in the course of a single 
human generation. Yet, software has experienced more notice- 
able growing pain. The gap between hardware — and software 
technology widens. 

B. TEE COST OF SOFTWARE IN DOD 

There are two general classifications of software within 
DoD. The first of these is that of the more-traditional, 
administrative type cf software used in business applica- 
tions. This type of software is typically supported by 
commercially available computer that can support a variety 
of applications, i.e. , Automatic Data Processing (ADP) 
systems. The second classification is embedded software. 
Embedded software is normally designed to operate as an 
integral part of ncn-ADP systems, such as DoD tactical 
systems. The most significant difference between these two 
classifications of software rest not in the development and 
maintenance practices, but rather in the frame work in which 
they are each procured. 

The procurement authority for Automatic Data Processing 
Equipment (ADPE) and its supporting software and services is 
vested in the General Services Administration (GSA) , as 
directed by Public law 89-306, 40 USC 759, the "Brooks 
Bill." Within DoD, ADPE is under the purview of the 
Assistant Secretary of Defense (Comptroller) . Weapon system 
software is under the cognizance of the Office of the 



Undersecretary of Defense (Research and Engineering). 2 
Although there is a distinct dichotomy of cognizant organi- 
zational structures regulating the acquisition of ADP and 
non-AEP software, the managerial and software engineering 
principles which govern each step of the software life cycle 
are, in fact, quite similar. Therefore, the common set of 
tools, methods, and methodologies advocated in this thesis 
apply to both ADP and non-ADP software. 

In writing this thesis, it was noted that the majority 
of available DoD guidance for the control and acquisition of 
software projects was in support of tactical systems, with 
the vast majority being authored for the United States Air 
Force. This is not suprising since it has been estimated 
[Ref. 2 : p. 7] that of the $12 billion that DoD will spend 
for software in 1985, over $10 billion will be for embedded 
software, with the U. S. Air Force accounting for approxi- 
mately half of the expenditures. 

Not only does embedded software represent the largest 
component of total software costs in DoD, it is also plagued 
to a proportional degree with many of of the software- 
related problems, which M.H. Lehman so aptly describes as 

”a motley collection of relatively isolated methodolo- 
gies and techniques, associated through an experience- 
based, but otherwise arbitrary sequence of 
much-discussed process phases”. [Ref. 3 : p. 3] 

At this point, it is important to recognize some of the 
some of the program characteri sties that add to the complex- 
ities of DoD’s embedded software. These include: [Ref. 4 : 
p. 77] 



2 Due in large part to the provisions set forth in the 
subsequent Warner Aamenment, the policies and procedures set 
forth in the Brooks Bill does not extend to the tactical 
software used in DoD weapons systems. 
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-- proqram size — often in excess of a million lines of 
code. 

— real-time operating requirements requiring response 
time in milliseconds. 

— programs must he flexible to accommodate changes in 
system evolution over an expected useful-life often 
in excess of twenty years. 

— guaranteed reliability due to the tight (and many 
times life- dependent) coupling between the system 
and its user or the population that the system is 
designed to protect. 

— programs are part of the universe which they model 
and control. 

As compared to other software applications, such as ADP 
or administrative computing, DoD mission-critical software 
is more complex, less understood, more unstable, and must 
operate in extreme environmental conditions. Yet it is 
essential that DoD software be reliable, adaptable, and 
affordable. To achieve these objectives, many problems, of 
both a technical and managerial nature, must be overcome. 
Symptoms of these problems include slippages in weapon 
delivery schedules, system failures, overbudget programs, 
and inflexible systems will be discussed at further lengths 
in Chapter II. 

DoD is recognized as the world's largest buyer of soft- 
ware. Based on various estimates in recent literature, it is 
calculated that DoD will spend approximately $12 billion for 
software in 1985 [Ref. 2 : p. 5]. Table 1 illustrates the 
percentage of total computing system costs of hardware as 
compared to software for all of DoD computing systems. 
Software cost reflect all aspects of the software life 
cycle, including: design, development, testing, operations, 

and maintenance. The ratio of hardware to software has 
reversed itself frcm 4:1 to that what is expected to 
approach 1:9 next year. [Ref. 2 : pp. 5-6]. 

The high cost of acquiring software has naturally 
caused concern in both DoD and the Congress. Literature 
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TABLE 1 

Cost Trends: Hardware versus Software 



(percentage of total cost) 

1985 

1955 1970 1979 (Estimate) 



Hardware 83 45 25 10 

Software 17 55 75 90 



abounds with studies and recommendations related to software 
development in DoD. There is not a shortage of sage advise. 
The need for improved managerial controls and software 
development practices has been recognized. 

C. PURPOSE AND APPROACH 

A major goal of this thesis is to present a consolidated 
review of major DoD efforts aimed at reducing software- 
related problems. Both management and technical issues will 
be addressed. This thesis makes no pretense that it provides 
the program manager with all of the technical background and 
controls needed to assure the timely delivery of guality 
software within budeget. Rather, it focuses on Key and 
"high payoff" issues involved in managing the acquisition 
and development of software. This thesis also addresses 
several DoD initiatives which promise to significantly alter 
the framework in which software is developed and maintained. 

Chapter II identifies many common problems associated 
with contracting for general computer software by Federal 
agencies. It also identifies major contributing factors to 
DoD weapon system software problems. A common denominator in 
the formulation of the many software problems is the lack of 
estimating expertise by which program measurements can be 
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defined. Chapter III discusses software metrics for defining 
guantifiable measurements. 

The delivery of good software is an implicit, but most 
elusive, goal in software acquisition. Chapter IV defines 
"good software" through a set of quality software character- 
istics. It also provides a series of controls to be utilized 
in the implementation of a quality assurance program. Major 
dividends from quality software are realized during the 
post-development phase of software, the maintenance phase. 
Chapter V analyzes the tangible and intangible costs of 
software maintenance, and addresses a number of software 
engineering principles through which the costs of mainte- 
nance can be greatly reduced. 

Chapter VI and VII review a number of software and 
computer- technology standardization initiatives to under- 
taken within DoD. Perhaps the most significant of these 
initiatives is the STARS (Software Technology for Adaptable, 
Reliable Systems) program. 

Finally, Chapter VII provides this thesis’ conclusions 
and recommendations. 



15 



II. THE SOFTWARE CRISIS 



"The problem of the 1970's was to reduce the cost of the 
electronic functions needed to store and process 

data. . .. 

"The problem of the 1980's is different. Now we must 
reduce the ccst cf electronic solutions; that is, 
reducing the cost you incur in using our devices to 
build a product. Solving this problem will reouire a 
shift from the component integration of the 1970's to 
concentration of system level integration in the 1980 *s. 

"We can now that about putting power of a mainframe CPU 
on a single chip. This buys you nothing as a customer, 
however, unless you can use that power. Hardware is 
computing potential; it must be harnessed and driven by 
software to be useful." [Ref. 1 : p.22] 

The preceding statement was made by the president of one 
of the largest manufacturers of computer hardware. It 
succinctly summarizes the shift in technological emphasis 
from hardware to software. 

A. PROB1EES IN SOFTWARE ACQUISITION 

To the casual observer, the successful management of a 
software development project may seem a simple process. All 
that is needed are (1) well-defined requirements, (2) real- 
istic cost and schedule estimates, and (3) the right quan- 
tity of personnel and hardware at the right time. In 
actuality, each of these elements seldom, if ever, happens 
by themselves, much less together. 

The management of software development projects have 
historically been plagued by a myriad of problems, both in 
the private and government sectors. 

1 . A GAO R epo rt 

Recently GAO reported to Congress [Ref. 5 : pp. 1 - 
84] a number of problems that Federal agencies have 

16 



encountered in contracting for computer software as an 
alternative to in-house development. Means for improving 
these deficiencies were also recommended. 

a. Scope of the GAO Report 

GAO sent questionnaires to 163 software 
contracting firms and 113 Federal project offices that had 
experience with software development projects. The purpose 
of the questionnaires was to attempt to identify common 
problems in software development contractual process and 
what, through hindsight, might have been done to prevent or 
improve development efforts. 

GAO examined nine cases of software development 
in detail, some of which had attracted GAO attention because 
they were known failures. Only one of these nine cases 
yielded a software product that could be used as delivered. 

The actual combined total development cost and 
time for the nine cases almost doubled the estimates of $3.7 
million and 10.8 years. 

b. Common Causes of Software Contracting 

The nine cases that were studied in detail 
illustrated many of the same causes of difficulties that 
respondents to the GAC’s questionnaires had identified. The 
most significant of these findings will now be described: 

— Federal agencies contract for software development 
with little specific guidance. 

Guidelines for software development promulgated 
by central agencies are primarily aimed at the technical 
aspects of software development. Very little guidance is 
provided in support of the contractual process. 

Basic responsibilities of the central agencies 
are set forth in the Brooks Act, Public Law 89-306. The 
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Office of Management and Budget (OMB) is prescribed general 
oversight of Automatic Data Processing (ADP) activities. 
Much of this responsibility has been delegated to the 
General Services Administration (GSA) and to the National 
Bureau of Standards (NBS) . GSA is delegated the responsi- 
bility for ensuring cost effectiveness in the selection, 
acquisition, and utilization of AD? resources. GSA's 
guidance for the management of ADP resources is contained in 
subpart 101-32 of the Federal Property Management 
Beg ulations. 3 Policies addressing the procurement of and 
contracting for commercially available software is provided 
in Federal Procurement Regulation 1-4.11. GAO's review of 
both of these documents revealed that there is very little 
actual guidance directed at the specific contractual manage- 
ment for engaging in custom software development. 

Although NBS is tasked with developing technical 
standards and guidelines, OMB has indicated that NBS is also 
responsible for investigating and assisting in software 
system developments. Although NBS representatives advised 
GAO that their responsibilities involved managerial and 
contractual activities for system development, MBS’ emphasis 
has been, and will continue to be, on the technical aspects 
of system development, such as the standardization of 
government-used Higher Order Languages. 

— Agencies overestimate the stage of their own prelimi- 
nary work before they contract. 

GAO found two primary reasons why agencies 
contract out for software development instead of doing it 
in-house. The first is that many of the agencies lack suffi- 
cient quantities of, or properly skilled, personnel to do 



3 As of 1 April, 1984, DoD regulations concerning the 
acquisition of ADPE resources and services previously 
contained in the Defense Acquisition Regulations (DAE) have 
been replaced by DoD supplements to the Federal Acquisition 
Regulations (FAR) ; specifically. Subchapter H, Part 70 of 
the DFAR. 
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the work. Secondly, the software is often needed sooner than 
it can be produced in-house. Often the initial steps of 
software development, such as requirement analysis, are 
started in-house prior to contracting out for the continued 
development of required software. Two common problems have 
been observed in this context. First, the agency may overes- 
timate the amount of work already achieved in-house 
secondly, the agency’s preliminary work that is turned over 
to the contractor may be inadequate requiring that it be 
done again by the contractor. 

Overestimating the stage of software development 
before releasing it to a contractor is likely to result in 
additional costs to the extent that any cost benefits that 
might have been gained from the development project are 
forfeited. It is critical that precise methods for measuring 
preliminary in-house work be used in order to achieve real- 
istic cost and time estimates. An accurate identification of 
the stage of system development is vital in order to prop- 
erly determine the type of contract to be utilized. If, for 
example, the agency has completed all the preliminary devel- 
opment stages required prior to the commencement of coding, 
then a firm-fixed price contract for the coding effort might 
be the most suitable. If, on the other hand, a systems 
detailed design has not been completed by the agency prior 
to entering into a contractual agreement, then a phased, 
cost-plus- fixed-fee type contract would likely be more 
suitable since the exact scope of future efforts is net yet 
known . 

If agency work that is passed on to the 
contractor is later found to be inadequate, or less than 
originally estimated, much of the work may have to be redone 
by the contractor. In doing so, there often is a tendency to 
attempt to save as much of the original work as possible in 
order to remain within the cost and time ceiling mandated by 
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the contract. This is likely to compromise the design of the 
new system, resulting in a less efficient system that 
mandates higher operating and maintenance cost for the 
remainder of its life cycle. 

-- Contracts fail to stipulate what constitutes satis- 
factory performance. 

Failing to stipulate what constitutes satisfac- 
tory performance by the contractor makes it difficult, if 
not impossible, to claim poor contact performance. 
Furthermore, it reduces the probability of a satisfactory 
end-product. Many disputes over contractor performance could 
be avoided if adequate system specif ications and testing 
criteria are identified in the contract. 

Other general requirements and constaints that 
can usually be identified at the start of a software project 
criteria for software expandability, documentation stan- 
dards, maximum computer resources allowable, maintenance, 
and program transfer capabilities. 

— Agencies quickly overcoramit themselves, and fail to 
adhere to strict phasing to control contractors. 

Phasing divides the development effort into 
logical and manageable work phases. One of the most effec- 

tive controls available to an agency is in the contractual 
identification of phases, coupled with manadatory agency 
review and approval following each phase as a precondition 
to the contractor's continuation of subsequent phases. Other 
advantages associated with phasing include: 

-- Identification of milestones and timetables to 
monitor the progress of the project, allowing for 
the initiation of corrective actions in a timely 
fashion. 

-- Systematic and orderly development of software. 

-- Control of funds based upon quality and accept- 
ability of contractor's work. 

-- Increased assurance that should development efforts 
are being used. 

-- Improved communication between the agency and the 
contractor leading to the increased probability 
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that the contractor fully understands the agency’s 
requirements. 

-- Completed phases provide an adequate base upon 
which subsequent phases can be built. 

-- Lack of agency management during contract 
execution. 

An excessive number of system changes were 
requested by the agencies in the cases studied by GAO. These 
agency-initiated changes ranged in scope from minor require- 
ment adjustments to re-resign of the entire system. Many of 
these changes requested and made during the latter phases of 
development and contributed significantly to cost and 

schedule overruns. 

Project managers should be aware of the need for 
a well-defined problem statement and the undermining effects 
that changes have cn software development. Changes, as 
compared to the original requirement specification, are not 
usually as thoroughly researched and may cause unforeseen 
and rippling effects on other parts of the system. The 
systematic and logical flow of contract phasing may be lost 
due to the need to modify work that has already been 
completed and approved, obscuring the visibility of the 
project's status. Furthermore, excessive changes make it 
difficult to hold the contractor accountable for the initial 
terms of the contract. 

— Agencies to not adequately inspect and test software. 

As depicted in figure 2.1, most of the software 
delivered in the cases studied was of poor quality. Reasons 
for this poor quality was evidenced in all phases of devel- 
opment. Quality assurance must be tied directly to the 
contractual process. Higher quality software can be 
obtained if the contractor maintains quality assurance func- 
tions in a number of software development areas. Specific 
examples of these areas include configuration management, 
testing, program design, documentation, and working tasks. 



21 



The latter of these area, working tasks, is a means for 
assuring that procedures are in affect for subdividing the 
total work effort into segments and assigning responsibility 
for the initiation and completion of work. 



NINE SOFTWARE DEVELOPMENT CONTRACTS TOTALING S 6.8 MILLION: 

WHERE THE MONEY WENT. 




SOFTWARE THAT COULD BE 
USED AFTER CHANGES 
(SI 98,000) 



SOFTWARE THAT COULD 
BE USED AS OEUVERED 
($1 19.000 out of S6 S million) 



Figure 2. 1 Value of Delivered Software 
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The GAO report concluded by stating the need for 
improvements in contracting for custom software development. 
Recommendations were made aimed primarily at GSA and NBS for 
both improvements is both procurement and technical areas. 

GAO further recommended that GSA and NBS work 
together in designing model contracts of various types. 
These contracts would have sample clauses for covering the 
withholding of payments, testing, etc.. Agencies would used 
these samples to extract those clause which best fit there 
particular requirements. 

The last recommendation that GAO made was that 
Federal agencies that extensively contract for software 
development "should train project managers in appropriate 
software, contracting, and management skills." £Ref. 5 : p. 
29] 

2. The Mu lti -source Unified Data Di st ributi on ( MUDD ) 

Report 

The MUDD Report [Hef. 6 : pp. 1-28] should be 
considered "required reading" by all present and future 
project managers overseeing software development. It is a 
case study of Navy software development practices. The 
report is based on over 30 interviews with of those respon- 
sible for the development of Navy systems. The year-long 
study uses the development of the fictional MUDD system 
under development to mirror many of the requirements of Navy 
tactical systems either in operation or under development. 
It chronicles and analyzes the decisions made on the soft- 
ware development effort. The MUDD Report concludes with a 
set of recommendations to Navy program managers for avoiding 
the pitfalls described in the report. 

The issues brought to light in the MUDD report are 
germane to those problem areas found in large and complex 
system development efforts which typify many DoD programs. 



23 



An adequate summation can not be given of the MODE report 
which can do it justice. It should be read in its entirety 
for a full appreciation of Weiss’ recommendations which are 
directed at problem areas that infest the fictional MUDD 
system development. Most of the recommendations center on 
various types of interfaces, such as the interface between 
the Navy and contractor, interfaces between people, inter- 
faces between and within systems, and interfaces within the 
Navy . 



3. EoD Wea pon S ystem Soft w ar e S tud y 

The John Hopkins University Applied Physics 
laboratory (APL/JHU) , in conjunction with the MITRE 
Corporation, conducted an extensive study of the management 
of weapon systems software under the auspices of the Office 
of the Secretary of Eefense [Ref. 27]. The MITRE and APL 
study team reviewed the findings of ten previous 
DoD- sponsored studies relating to software. The MITRE 
Corporation concluded that the "major contributing factor to 
weapon system computer software problems was a lack of 
discipline and engineering rigor applied consistently to the 
software acquisition activities." [Ref. 7 : p. 50] Other, 
more specific, findings of included in the MITRE/AFL study 
included the following: [Ref. 7 : pp. 50 -51] 



--Frequent contributors to software cost and schedule 
growth include: (a) poorly formulated initial software 

requirements: (b) changing requirements and require- 

ment growth during the development phases; (c) false 
starts and need to educate involved organizations 
before useful output can be obtained; (d) inefficient 
use /proliferation] of already existing resources; (e) 
inefficient testing and verification tools and 
methods; and (f) improper use of standards and 
guidance documents in specific procurements. 

— There is a general need for better identification of 
software terms, measures of software qualities, and 
the methods for measuring them. 

--Software technology improvements particularly aimed at 
developing a software engineering discipline are being 
made by industry, academia, and the services but 
require application to real military systems (in 
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addition to laboratory or experimental systems) for 
evaluation and confirmation. 

The study resulted in a series of 17 recommenda- 
tions, each of which was directed at a specific problem 
area. A sairple of these recommendations included: [Ref. 7 : 

pp. 50 - 51 ] 



— Specify that major computer software involved in 
weapon system development be designated "configuration 
items" and be deliverable during full-scale develop- 
ment. 

--Use top-down design, i.e. specify the use of modular 
software architecture ana an orderly, phased design 
approach that defines the higher levels of the program 
ana then progresses to design and test successively 
lower levels. 

— Require the contractor to apply a highly disciplined 
set of engineering practices. 

--Establish a common set of requirements and criteria to 
be applied... by all services. 

— Prepare a series of handbooks of guides covering 
important aspects of software acquisition. 



While extensive progress has been made in DoD toward 
addressing many of the problem areas noted in the preceding 
studies, much work remains to be completed. Specific correc- 
tive actions that have been adopted, or which are presently 
in the formulation stages, will be covered in this thesis, 
part icu lar ily in the chapters addressing standardization and 
the STARS software initiative. 
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III. MEASURES OF CONT ROL 



A. BACKGROUND 

"You can't control what you can't measure." [Ref. 8 : 

P. 3] A disparity exists between the software manager's 
definition of what constitutes a project's success as 
compared to the user's perception of the same project. With 
software projects resulting in utter failures or cost over- 
runs of two, to three, times the original estimate, software 
managers often consider their projects a success if overruns 
are kept below 30% and when "most" of the delivered lines of 
code are considered "usable" by the end-user. 

DeMarco [Ref. 8 : p. 4] writes that many projects fail 

eventhough the project managers have excelled in those 
characteristics that he associates with good management. 
These characterist ics include: 

— project staff members that are highly motivated 
--clear understanding of the issues 
— adequate grasp of relevant technologies 
--evident capability in the political sphere 

Demarco attributes the failure of these project managers to 
the fact that they have simply failed to meet the original 
expectations of the project. He is convinced that in often it 
is not the fault of the project team, but rather the fault of 
"inflated and unreasonable expectations." When expectations 
exceed what can be delivered, the project is doomed to 
failure. 
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E. CAUSES FOR POOR SOFTWARE ESTIMATING 



Estimating is at the core of the difficulties 
surrounding software projects. Feedback is essential for 
control. Feedback provides a basis for comparing the actual 
project’s progress against original expectations. These 
expectations were formulated on estimates. Main causes of 
poor software estimating are as follows: [Ref. ® : pp. 9 - 
17] 

1 . l ack of Estim ating Expertise. 

The average software manager will typically rate 
himself/herself well below average as an estimater. The 
underlying reason for this is simply lack of estimating 
practice or experience. 

The amount of actual estimating that a typical soft- 
ware manager is involved in will normally take up less than 
3 % of his/her time. Most software projects may call for 
estimates at their beginning, and maybe once-a-month or 
prior to management review thereafter. 

2 • Bia ses in Es t imatin g 

Personal biases create a strong tendency to underes- 
timate one’s own potentials. However, when objectively esti- 
mating another’s potential, then most of these biases are 
minimized. DeMarco suggests an obvious approach to avoid 
this phenomenon by stating 

"Whoever does the estimating for a project must be 
someone whose entire ego involvement is in the quality 
of the estimate, rather than in the project itself...." 
[Ref. 8] F J 
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3 • Poor Und erst a ndin g of Wh at Estimate Keans 

At the very heart of probability theory is the esti- 
mation of "odds" of the occurrence of a certain event. Yet 
software estimates are often void of any explicit probabi- 
listic assessment which may govern them. This observation is 
closely linked to preceding subsection on the personal 
biases involved in estimating. Should a software manager be 
asked the probability of finishing a project, say, 20 % later 
that s/he originally estimated, an answer (right or wrong) 
will freely be given. On the other hand, should the same 
person be asked the probability of completing the project 
earlier than originally estimated, the estimater will likely 
give it a zero probability. This represents what DeMarco 
[Ref. 8 : p. 14] terms 

Defa ult Definition of "E st imate" 

An "estimate" is the most optimistic prediction that 
has a non-zero probability of coming true. 

Instead, DeMarco [Ref. 8 : p. 22] proposes that an estimate 
should be defined as "a prediction that is equally likely to 
be above or below the actual result." This definition, by 
itself, does not solve the estimating problem. It does, 
however, offer a basis from which to start examining meas- 
ures and other components of estimates which will be covered 
in this chapter. 

4. Estimates as Easis for Incentives 



Cften estimates are used to establish gcals for 
performance. When used in this manner, the software manager 
is likely to establish estimates on previously established 
goals. To serve as a motivational tools for the development 
staff, the goals are set at unattainable levels. 
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C. SOFTWARE METRICS 



The first part of this chapter has don 
point out many of the ill-fated approaches 
tionally been used tc control software, 
were principally qualitative in nature, 
mathematical basis. Yet, intuitively, a d 
exists between software quality and quan 
characteristics such as modularity, s 
paths. As such, software metrics have' been 
authors as a preferred means for derivi 
estimating process. 

This section will examine two of the 
ries in software metrics that have grown 
tive years of software engineering: (1) H 

Science Theory, and (2) McCabe’s Co 
Appendix B provides sample algorithms 
formulas for each of these two theories. 
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1 . Ha l ste ad 1 s S oftware Scie nce 



The first set of metrics to be reviewed w 
oped by Maurice K. Halstead [Ref. 9], Instead 
’’lines of code" (LOC’s) to describe the size of 
Halstead breaks each line down into a series, or 
symbols. Each of these symbols can be classified 
an "operator" of as an "operand." An operand is 
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symbol or group of symbols that identifies the constants or 
variables that the module uses to implement its algorithm. 
An operator is a single symbol or group of symbols which 
affects the value of the operand. Operators also impact the 
sequence in which the operation takes place. 

Criticisms concerning Halstead’s theory of measuring 
through the use of operators and operands were quickly 
registered. The majority of Halstead's work evolved around 
algorithms drawn from Algol and FORTRAN. In other algo- 
rithmic languages, the definitions of operands and operators 
are not nearly as clear. Halstead also omitted declarations 
and comments from his calculations--a significant portion of 
the widely-used COBOL language. Other studies, however, have 
shown that the additional declarations and comments actually 
brought the estimated program length closer in line with the 
actual, completed program. In any case, it is important to 
identify the operands and operators of an algorithmic 
language to establish consistency. This function can often 
be determined by a compiler, through which the operators and 
operands are explicitly defined in the final machine 
language product. Questions abound on the derivation of 
Halstead's formulas. The validity of his experiments has 
been questioned because of the small sample size, the small 
size of the programs used, and the subjects used were 
college students vice experienced programmers. 

Halstead proposes that each language can be categor- 
ized by a language level, 1, which will vary among 
languages. The variances are closely linked to the level of 
abstraction by which a procedure can be specified. Halstead 
assigns a constant language level for a particular language, 
which is in contrast by to the recent works that show that 
language level is a resulting product of both the language 
and the programmer. Table 2 provide the language levels 
values that have been empirically derived for five common 
languages [Ref. 1: p. 166]. 
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TABLE 2 

Language Level Values 



Language: 


Mean 1 


English pose 


2.16 


PL/1 


1.53 


ALGCL/66 


2.12 


FORTRAN 


1.14 


Assembler 


0.88 



2 • Mc Cabes * Com plex ity Measure 

In his article, [Ref. 12 ; PP. 308 -320] McCabe 
proposes a complexity measure of software which is based 
upon the control flow representation of a program. Through 
the analysis of several FORTRAN programs, he illustrates a 
high correlation between the intuitive complexity of a 
program and his proposed graph-theoretic complexity measure. 

McCabe’s software complexity measure is preported to 
measure and control the number of paths through a program. 
The primary difficulty is this regard is manifested through 
backward tranches which may possibily result in an infinite 
number of paths during program execution. Conseguent ly , 
using a path count to measure program complexity is inprat- 
ical. However, the complexity measure can be defined in 
terms of basic paths, that when taken in combination, will 
measure every possible path. 

As compared to Halstead’s metrics, McCabe’s 
complexity measure can be applied during the earlier stages 
of software development since it is not dependent on the 
measurement of code. The cyclomatic complexity measurement 
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provides an evaluation tool by which "goodness" of a module 
can t € reviewed following its detailed design [Ref. 1 : p. 
169 ]. 



D. SOFTWARE COSTING 

The role of software in the military and private sector 
has grown considerably during the past decade. During the 
infancy cf computing, software cost amounted to only a small 
percentage of the overall computer system. Today, software 
is the most significant portion of most computer systems. 
Accurate estimates cf software development cost seldom 
occur, with the final costs normally running considerably 
higher. There are two fundamental problems which make accu- 
rate estimates of software development costs most difficult 
[Ref. 13 : p. 45]. These are: 

— the high risks and uncertainties involved in software 
development 

— the lack of a quantitative data base for previous 
cost estimates and final costs. i.e., "lessens 
learned" 

In spite of these significant problems, cost estimate are made 
and will continue to be made with varying degrees of accuracy 

This section will describe three current methods of cost 
estimating and provide a table for their comparison based on 
application [Ref. 13 : p. 15 - 17]. 

1 . Ana log y 

This method estimates the costs for a new system 
based upon the the costs of a similar system. The cost esti- 
mate is adjusted to compensate for any differences between 
the two systems being compared. The analogy method is fairly 
simple provide accurate cost data for the existing system is 
available and the development methods and resources are 
similar. 
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2 . Dec omp os iti o n 

As the method name suggests, a system is broken down 
into components and subcomponents until the level of decom- 
position makes it possible to estimate the costs fairly 
accurately. One approach of decomposition uses the analogy 
method previously described. In this approach, the process 
of decomposition is effectuated until the resulting level of 
decomposition can be compared with a similar component which 
already exists. A second approach of decomposition divides 
the system into components for which a level of effort can 
intuitively be estimated for each kind of activity that is 
needed to produce that component. This latter type of decom- 
position normally depends heavily upon the technical knowl- 
edge and experience of the estimater The preassumption that 
underlies this method of cost estimating is that the costs 
for small systems, or components, can be accurately made. 
The total system is perceived as the aggregate total of its 
subsystem. 

3 . P arame tric M odels 

As with the analogy method, the parametric model 
approach to cost estimating is also heavily dependent on the 
accumulation of past and accurate cost data for software 
development. Analyses of cost data permits the identifica- 
tion of cost variables and a quantification of their rela- 
tionship to cost. Any new cost estimate can be derived by 
estimating the assigned values of the cost variables. Once 
this is done, the cost can be calculated using the equations 
which express the cost estimating relationships. The advan- 
tages of this method is that it allows for a rapid determi- 
nation of cost estimates, using parameters whose values can 
be easily modified. 
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Table 3 [Ref. 13 : p. 16] provides a comparison of 
the three cost estimating methods discussed. Combinations of 
two, or more, methods may be used together, or separately to 
test the validity of an estimate. Resultant differences are 
adjusted to arrive at a reasonable estimate 



TABLE 3 

Comparison of Cost Estimating Methods 



TYP E 

Analog 

Decomposi tion 



Parametric 

Models 



DES CRIPTION 

Compare to prior system 

Divide into parts, 
activities 

Equations based on prior 
data about cost relation- 
ships 



TYPE 



Analog 



GOOD FOR 



Decomposition 



Parametric 

Models 



similar systems with 
similar resources, 
development process 

resource allocation 
unique systems 

rapid estimation 
estimater's inexperience 
in software development 
estimating risk 



TYPE 



BAD FOR 



Analog 

Decomposition 



Parametric 

Models 



unique systems 
different environments 

initial estimates with no 
design 

rapid estimation 

systems different from 
data sample 

poorly correlated data 
base 
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Automated costing systems provide another option for 
estimating. In these automated systems, the character istics 
of the development organization, such as the staff's experi- 
ence level, and characteristics of the software to be devel- 
oped are described. Cost estimates are derived from this 
input data. As with the other three manual methods, the 
derived cost data will only be as good as the empirical data 
upon which it is founded. If no historical data exists, 
then the validity cf the cost estimates is, indeed, 
questionable. 

E. CHAPTER SUMMARY 

McCabe's and Halstead's software metrics remain a 
controversial topic. But they do represent a revolutionary 
approach toward providing software managers with quantita- 
tive functions for estimating many heretofore elusive char- 
acteristics of software. The validity of Halstead's 
experiments have yet to be significantly tested. For those 
tests that have been performed, the size of the programs 
were generally small, and the subjects were college students 
vice professional software developers. 

As compared to Halstead's metrics, McCabe's complexity 
measure can be applied during the earlier stages of software 
development since it is not dependent on the measurement of 
code. The cyclomatic complexity measurement provides an 
evaluation tool by which "goodness" of a module can be 
reviewed following its detailed design. Despite the criti- 
cisms that normally abound the proposition of new theories, 
both Halstead's and McCabe's metrics represent a giant leap 
toward adding quantitative measurements to a discipline that 
has defied them. 

The military's justified and growing concern over 
frequent cost overruns for software development is forcing 
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changes in both the management and develo 
As such, new requirements and changes ca 
will provide a more uniform and better con 
software development. This chapter has a 
the most common software cost estimating m 
program. Chapter VII, addresses a DoD- 
initiative which will significantly alter 
software development efforts for DoD weapo 
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IV. QUALITY SOFTWARE 



"Software correctness remains the most elusive goal of 
computer science. As a result, software is the most 
unsafe, the least understood, and the most expensive 
component of total computer system cost. In contrast, 
cost of computer circuitry have shown a dramatic 
decrease, especially in the past 15 years, and computer 
hardware cap-ability has improved." [Ref. 14 : p. 1b] 



A. BACKGROUND 



The preceeding quotation was taken from an article 
authored by the Deputy Under Secretary of Defense (Research 
and Advanced Technology) , Dr. Ruth Davis. It expresses the 
concerns shared by many DoD top officials relating to the 
both cost and safety risks associated with the development 
of today's computer systems. 

As a percentage of total computer system cost, it is 
generally known that the cost of hardware has decreased 
dramatically over the the past 15 years while the cost of 
software has steadily increased. Today, software represents 
approximately 90i( of the total computer system [Ref. 14 : p. 
18], There are two basic reasons to explain the change in 
the cost ratio between hardware and software. These are: 
[Ref. 15 : pp. 55 - 56] 



(1 ) Size: Today’s software programs are an order of 

magnitude larger than they were two decades ago. 
This can be attributed, in part, to the increase in 
size (and simultaneous decrease in the cost) of 
onboard memory. An adaptation of Parkinson's law 
suggests that program instructions will continue to 
increase until the limits (and frequently beyond) 
the available core is fully utilized. 

(2) C om pl exit y : As nature of the applications being 

automated today are considerably more sophisticated 
than those applications of yesteryear. Both mili- 
tary and commercial survival strategies are 
becoming increasingly dependent on maintaining the 
competitive edge in computer superiority. 
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Software has become a primary vehicles for solving many 
of the new and changing problems facing the military. In 
many cases, changes to software is often viewed as an effi- 
cient and expedient way to solve a variety of emerging prob- 
lems or threats facing DoD without having to change the 
existing hardware. let the virtues of software are often 
outweighed by its associated problems as described in 
chapt-er II. 

It is not suprising that DoD has identified software as 
the most significant factor in determining the total cost of 
computer- based systems over their life cycles. Numerous 
studies have been conducted which show software quality as 
one cf the most significant factors determining the life 
cycle cost of software. This chapter will present many of 
the characterist ics cf software quality and the means to 
achieve them. 

B. DEFINIBG SOFTWARE QUALITY 

Defining guality software is, in itself, a task. There 
are as many definitions of what makes software "good" as 
there are authors that write about it. Yet these definitions 
are not mutually exclusive. Each author has his own ideas of 
what the principle character istics of guality software are, 
and each is right. Defining quality software is as difficult 
as defining the virtues of mankind. Air Force Regulation 
(AFR) 74-1, the "Air Force Quality Assurance Program," 
broadly and sensibly defines quality as "The composite of 
material attributes including performance." Other, more 
specific, definitions advocated by many of the "gurus" of 
software engineering will now be discussed. 

Pressman [Ref. 1 : p. 148] suggests that good software 

has three essential qualities; 

— the software works according to the specified 
requirements — being as fast, efficient, and as func- 
tional as needed. 
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-- the software is maintainable — it can be diagnosed and 
modified without great difficulty. 

-- the software is more than merely lines of code--it 
includes all the supporting documents to ensure that 
the first two qualities are achieveable. 



According to Pressman, good software is based upon good 
design, and good design can be gauged by applying a number of 
software engineering measures and heuristics. 

DeMarco [Ref. 8 : pp. 198 - 200] prefers to define soft- 
ware quality as "the absence of spoilage" [Ref. 8 : p. 200], 
with the term "spoilage" meaning the amount of effort 
required to find and remove faults introduced during the 
software development process. Equating this amount of effort 
to its commensurate cost, Demarco provides a formula to 
guantify software quality: [Ref. 8 : p. 200] 



Quality 



Summation of Defect Diagnosis and Correction Cost 



Program Volume 



In which Program Volume is measured per thousand 
lines of executable code (KL0EC) 



C. CHARACTERISTIC .OF SOFTWARE QUALITY 

One of the most comprehensive and significant works 
written to provide a framework for assessing the issues 
associated with software quality is found in the study 
conducted by Boehm, et al. , titled Characteristics of 
Q ua lity Softwa re [Ref. 16]. This section will present many 
of the highlights reported by this study. 

In developing a methodology for the assessing the 
quality cf software products, the authors concluded that 
"calculating and understanding the value of a single overall 
metric for software quality may be more trouble than it is 
worth." [Ref. 16 : p. 3-2] A major problem in developing a 
single metric for gauging the quality of software is that 
many of the characteristics of software are in conflict with 
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one and another. For example, requiring a high degree of 
software portability is achieved at the expense of software 
efficiency. Code conciseness is at odds with maintain- 
ability, under standability, and so forth. As such, the study 
developed a relational set of important software character- 
istics which were reasonably exhaustive and non- overlap- 
ping. This set of characteristics would serve to define a 
working context for collection and formulation of a set of 
candidate metrics used to assess the degree to which the 
software possessed the respective characteristic . Figure 4.1 
shows the resulting characteristic set and their hierarchial 
inter relationshi ps [Eef. 16 : p. 3-19]. Definitions for 
each of the represented characterist ics is provided in 




Figure 4.1 Characteristics Tree 
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appendix A. The characteristics depicted in Figure 4.1 are 
categorized in three hierarchial levels. The higher-level 
structure is oriented toward accommodating various user 
needs and priorities for a software product. For example, 
"As-is" utility is analogous to the "black box" under- 
standing of a system; the user is concerned with only the 
inputs and outputs of the product and need not understand 
the its internal code, nor how to -modify or test it. If the 
product is going to be changed by the user, then maintain- 
ability requires that the user be able to understand, 
modify, and test the product. 

The lower-level structure depicts those primitive char- 
acteristics, which, although strongly differentiated from 
each other, "combine into sets of necessary and sufficient 
conditions" [Ref. 16 : p. 3-25] to define the intermediate- 
level characteristics. The primitive characteristic provide 
the foundation for formulating the metrics used to quantifi- 
ably measure a software products relative possession of 
those characteristics described in both the high — and 
intermediate layers. 

D. QUALITY ASSURANCE 

The preceeding section described many of the attributes 
associated with gocd software, as well as their interrela- 
tionships. The purpose of this section is to offer a frame- 
work through which quality software can be achieved through 
planning, specification, and monitoring of quality assurance 
(QA) activities. 

The purpose of software quality assurance, in short, is 
to assure the ultimate quality of the delivered software. A 
formal definition of quality assurance is provided by AFR 
74-1, which defines it as: 



" A planned and systematic pattern of all actions neces- 
sary tc provide adeguate confidence that material, data, 
supplies, and services conform to established technical 
requirements and achieve satisfactory performance." 



Another definition for quality assurance is offered by Pfau 
[Ref. 17 : p. 2] who also helped remove some of the subjec- 
tivity that surrounds the term "quality" by stating: 



"Quality assurance is the name given to the activities 
performed in conjunction with a software product tc 
guarantee the product meets the specified standards. 
These activities reduce doubt and risks about the 
performance of the product in the target environment." 



Both cf the above definitions are reflective of the direc- 
tion that QA has taken over the past two decades toward a 
total life-cycle perspective. This evolution of QA has been 
divided into three separate generations [Ref. 18 : pp. 2 - 
4]. It is important to understand the differences in these 
generations in order to avoid the serious pitfalls implic- 
itly and explicitly expressed in the first two. 

First G ene ra tion — Te st- O riented QA: This QA generation 
basically equated QA to software test programs. Tests plans 
and procedures, types of test, and methods of formal verifi- 
cation of performance/design requirements were all essential 
to the testing activity. 

The obvious and major pitfall of the test-oriented QA 
generation is that "you don't test quality into a software 
product." [Ref. 18 : p. 2] Even though testing facilitated 
the discovery of deficiencies, the discovery normally took 
place too late in the development process to allow their 
relatively inexpensive resolutions. 

Second Generation — D e velopment-O riented QA: Due to the 
inherent failure mechanisms built into test-oriented QA , 
corrective actions were taken by an attempt to make the 
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developing contractor responsible for the quality shortcom- 
ings or the products they produced. This was done by 
assuring that the software delivered under contract fully 
complied with the requirements of the contract. 

The pitfall to this Q A approach is as limited as the 
contracting officer technical knowledgeable in the bread 
discipline of software engineering. Contract delivered what 
was specified, nothing more. 

Third Generat io n"Li fgz£vcle-Or i en ted 21 : In this 

generation, Qh is built into the software from "day one." 
The effort is properly focused on the early definition 
phases for planning and specifying contractual provisions 
concerning software attributes. Figure 4.2, [Ref. 1 : p. 25] 
illustrates the cost impact of introducing changes during 




Figure 4.2 Cost Impact of Changes 

various phases of the software life-cycle. Emphasis is 
placed on the clear definition of those software character- 
istics that were discussed in the first section of this 
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chapter, such as maintainability and portability, which have 
a significant affect cn the quality of the product over the 
system's life-cycle. The importance of the life-cycle- 
oriented Q A approach and its impact on life-cycle costs 
following software development and implementation is 
discussed at length in the chapter on software maintenance. 

E. IMPLEMENTATION OP A SOFTWARE QUALITY ASSURANCE PROGRAM 

Ihe preceeding section addressed the definition of soft- 
ware guality assurance, as well as its evolution to the 
present life-cycle-criented perspective which recognizes 
that to achieve the highest guality of software it is neces- 
sary to include quality checks throughout all phases of the 
software life-cycle. 

This section will discuss the military standards for the 
implementation of software guality assurance (SQA) programs 
in defense contracts. The successful implementation of these 
programs will provide early visibility and managerial 
controls to detect, report, analyze, and correct software 
deficiencies. Although the focus of this discussion will be 
on defense contracts, the methods a'ddressed herein may be 
equally beneficial to in-house development efforts. 

The two most significant military standards affecting 
the establishment of SQA programs for defense contract are: 
(1) MIL-STD-52779 (A) , "Software Quality Assurance Program 
Requirements," providing the basic elements required in an 
acceptable SQA program, as well as customer evaluation 
criteria, and (2) MIL-STD-1679, "Weapon System Software 
Development," providing detailed software development stan- 
dards for the entire weapon system software development 
process. Both the software manager and procuring agent 
should he familiar with their contents since, together, 
these standards provide an effective means to evaluate any 
software development program [Ref. 19 : p. 108]. 
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In their article [Ref. 19], Dobbins and Buck discuss 
five areas of control which follow the typical chronology of 
software development. These are: (1) procuring agency evalu- 
ation, (2) design inspection, (3) code inspection, (4) test, 
and (5) library controls. The remainder of this section will 
address each of areas separately. 

1 . Pro cur ing Agency Ev aluation 

From both a cost and effectiveness standpoint, the 
conseguences are too important to accept at face value the 
claims that a strong SQA program exists in their organiza- 
tion. There must exist some means to evaluate the potential 
contractor. Major guality items must be addressed as early 
as possible in the planning process prior to the Request for 
Proposal (RFP) preparation. These guality items should 
include those attributes considered as an integral part of 
the software design, development, test and evaluation, and 
maintainability issues. Table 4 [Ref. 18 : p. 33] provides a 
number of factors with which to evaluate bidders’ responses 
to the RFP process. 

Cften the program manager and procurement agency 
will have insufficient experience and technical background 
to properly identify essential QA issues needed for inclu- 
sion in the RFP nor the means or time to evaluate the 
contractor proposals. In these situations there are alterna- 
tives resources available to evaluate the contractor. The 
first of these is the Defense Contract Administrative 
Service (DCAS) . The program manager can hire the services of 
software engineers acquainted current military QA standards. 
Depending on the end-application of the software product, 
there are other government organizations through which 
assistance can be sought. Other alternatives include hiring 
the services of a commercial contractor or consulting firm. 
Regardless of the resource used, a sound means for 
contractor evaluation and selection is essential. 
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TABLE 4 

Evaluation Factors in Bidder Responses 






FACTORS: 

Completeness 



Scope 



BIDDER EVALUATION CHECKLIST: 

Dees bidder's response cover all 
area as requested in the RFP? 

Is the scope of the bidder's 
response consonant with the 
project's objectives and the level 
of detail in the RFP ? 



Compliance 

Documents 



Are all compliance documents 
regarding the software design 
identified? 



Design Review 
Eoard 

Event Sequences 



Problem 

Reporting 

Action Item 
Follow-Up 



Past Experien- 
ces/Resources 



Dees the bidder propose to have 
design Review boards? 

Are the reviews of software design 
done in proper sequence? 

Dees the bidder propose to formally 
identify all design problems? 

Dees bidder provide assurance for 
effective follow-up of all action 
items resulting from reviews? 

Ehat experience does the bidder 
have with QA? Does he have a good 
resource base? 
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2 . Design Insp ec tion 

Although MIL-STD-1679 specifies that "walkthroughs" 
should be used as the means to collect statistics during the 
design phase of software development, the process has 
evolved to the more formal procedure termed "design inspec- 
tion." The difference between the two approaches is one "of 
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rigor, riot intent" [Bef. 19 : p. 110]. The walkthroughs 
were informal examinations of the software product by its 
authors’ technical peers. Little documentation was kept, and 
no training requirement placed on the participants of the 
walkthrough. 

As with the walkthrough, the design review is a peer 
inspection process performed by teams that inspect one 
another’s work. Unlike the walkthrough, the design review 
is a formal process in which records are kept, and partici- 
pants undergo considerable training requirements. It is 
conducted when the design is completed, prior to the actual 
coding effort. The inspection team is led by a moderator. 
The ideal moderator is not only trained in the technical 
aspects of software engineering, but in the psychological 4 
aspects of software development. 

The moderator promulgates required inspection 
material to the team in advance of the inspection. Each team 
member reviews the material and records comments before the 
inspection meeting. During the meeting, discussion is 
reserved for major error, i.e., those errors that will 
prevent the program from functioning properly. Minor errors 
simply recorded for subsequent correction. If more than 5 % 
of the program design must be changed due the errors, the 
entire design will be reinspected. Otherwise, the moderator 
will assure that the errors discovered during the design 
inspection are corrected before proceeding into the next 
phas e . 



4 Eor an more in depth discussion of the psychological 
aspects involved in software development, the reader is 

referred to Gerald Weinberg's book Tne Psychology of 
Computer Pr ogr a mmin g . 
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3. 



Code I ns £ec t ic n 



Programming coding may begin only after successful comple- 
tion of the design inspection. HIL-STD- 1 67S requires top- 
down structured programming and identifies the specific cede 
constructs allowed. Figure 4.3 [Ref. 20 : p. 62], illus- 
trates the five basic code structures, each having a single 




Figure 4.3 Basic Code Structures 
data entry and exit. 

Cnee the program is coded and successfully compiled, 
it is inspected. The process for inspecting co ie is nearly 
identical to that discussed for design. Not only is the code 
inspection a method cf discovering coding errors, but just 
as importantly it assures that the code alheres to the 
approved design. 
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4 . Test 

Software testing accounts for the majority of technical 
efforts expended in software development. Its objectives are 
to uncover software errors, and to provide assurances that 
the software performs its technical and operational 
requirements. 

An effective SQA program must start at the front end 
of software development, with the requirements specified in 
the EFP, addressing the totality of the testing tc be 
performed. Three measures of software testing relating to 
the RFP include: [Ref. A059H068 : p. 19] 

(1) The analysis cf software requirements for test- 
ability. 

(2) The identification of the contractor’s software 
testing activities as part of his Software QA 
Program. 

(3) The review of test documentation and certification 
cf test results. 

Testing requirements specified in MIL-STD-1679 
require that the system software do more than just just meet 
the specif icatio ns. Software must also be subjected to a 
third-party 5 "stress test," in which the program is judged 
unsatisfactory if the program execution can be stopped for 
whatever reason. To achieve a degree of software quality 
sufficient enough to pass this type of testing, it is vitu- 
ally essential that the software development program incor- 
porate programs of error detection and prevention well in 
advance of the actual testing period. 



5 As defined in MII-STD-1679, the third party is neither 
the contractor nor the procurement agency. 
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5 • li bra ry Cont ro ls 

A key element in any SQA program is the software 
library which provides visibility and control of the prod- 
ucts documentation and programs. Among the mandatory 
controls stipulated by MII-S-52779 (A) is the control to 
prevent unauthorized access. Other essential activities in a 
software library are the documentation and program storage, 
and retrieval and change processing. MIL-STD-1679 requires 
a Software Change Control Board (SCCB) , which must authorize 
any changes to the controlled library. 

I. PARTING COMMENTS 

The underlying goal of software development is to 
deliver quality software. In doing so, it is vital to 
examine the characterics of quality, and their interrela- 
tionship, within the context of the user needs and ultimate 
application of the program. To understand the characteris- 
tics of quality software is to understand the founding prin- 
ciples of software engineering. To produce quality software 
is much more. The implementation of a software quality 
assurance program is the vehicle through which these princi- 
ples are applied and the goal of software development 
realized. 

The benefits derived from quality software support the 
saying that "quality is free." But more importantly, as will 
be addressed in the next chapter, future cost-avoidance 
during the maintenance phase leaves no practical alternative 
to acceptance of only quality software. 
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V. SOF TWA RE MAI NTEN ANCE 

This chapter deals with the last phase of the software 
life cycle. Canning [Ref. 21 : p. 2] appropriately categor- 
ized software maintenance as an "iceberg,” initially 
revealing c'nly a small portion of maintenance requirements, 
but hiding an enormous potential for future problems and 
costs under the surface. With few exceptions, computer 
programs are always changing in order to correct latent 
errors, add enhancements, and seek performance optimization 
of the software. A succinct definition for maintenance can 
be given as "that activity which is concerned with making 
changes to software for the purpose of improving or 
correcting the software." [Ref. 22 : p. 2] Maintainability 
is defined 22 as "a property of software which makes the 
maintenance activity easy to perform, i.e., changes tc the 
software are easy tc incorporate and do not lead to new 
errors in the software." This chapter will primarily address 
issues of maintainability which by necessity must be consid- 
ered during all phase of the software life cycle. 

A. CATEGORIZATION OF MAINTENANCE ACTIVITIES 

Maintenance is much more than just fixing errors that 
escaped detection during the pre-delivery tests and evalua- 
tions. Maintenance has been categorized [Ref. 1 : p. 323 ] 
into four activities that take place after the program is 
released for use. These are corrective maintenance, adaptive 
maintenance, perfective maintenance, and preventive mainte- 
nance. Each will now be described. 

Correct ive m aintena nce is the process that includes the 
diagnosis and correction of latent errors that avoided 
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detection prior to the implementation of the program. It is 
impractical, if not impossible, to exhaustively test complex 
programs in order to guarantee 100% error-free software. 

Adaptive ma inte nance are those modifications made to the 
program as a result of changes to the enviroment in which 
the program must operate. As an example, it is often quicker 
and less expensive tc modify software rather than the hard- 
ware in order to modify weapon systems to satisfy new threat 
situations. 

Per fective mai nte nance is the process used to accommo- 
date recommendations for new capabilities, changes, and 
general enhancements requested by the user of system 
prog rammer. 

Preventive main tenance takes place when software is 
changed in order to improve its future maintainability. This 
type of maintenance remains a rare practice in software 
engineering . 

Based upon a study of 487 software development organiza- 
tions by Lientz and Swanson, as summarized in reference 1, 
50% of all maintenance is perfective. Corrective — and 
adaptive maintenance account for 21% and 25-%, respectively. 
All other types of maintenance account for only 4%. 

B. TANGIBLE 3AINTENANCE COST 

Although considered by many software engineers as the 
less glamorous and unexciting phase of software development, 
maintenance accounts for the majority of the dollars spent 
throughout the software life cycle. The cost of maintenance 
has shewn a dramatic and steady increase over the past two 
decades. As depicted in figure 5.1, one author [Ref. 1 : p. 
326 ] estimates that maintenance cost as a percentage of the 
total software budget will have grown from 35-40% in 1970 to 
70-80% in 1990. 
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Figure 5.1 Maintenance Cost as Percentage of Budget 

Although empirical data is available to account for 
total software life-cycle cost allocatable to maintenance, 
maintenance costs are very difficult to estimate in advance. 
It is known, however, that maintenance costs are often 
dramatically underestimated by both industry and government 
suring the pre- deployment phases of system acquisition. To 
illustrate this point, Boehm [Ref. 23 : p.127] estimate! 
that it took 530 to develop a line of code (LOC) , but the 
cost per LOC skyrocketed to 34000 in the maintenance phase. 
Although $4000 per ICC may seem unreasonably high, it is not 
unusual to incur such high costs for maintaining mission- 
critical software in CoD weapon systems. 

Although there is not a set of universal factors that 
can be applied to all software development projects to accu- 
rate estimate the relative cost of program modification in 
each of its life cycle phases, figure 5.2 illustrates the 
exponential rise in maintenance costs in each of the phases. 
[Ref. 24 : p.14] 
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Figure 5.2 Life Cycle Maintenance Costs 

It is apparent that there is more potential for real- 
izing life cycle cost savings by devoting more planning 
during the earlier phases in order to minimize the reguire- 
ment for maintenance during subseguent phases. One of the 
primary reasons for the high cost during the later phases is 
due tc the "domino etfect" of changes that must be promul- 
gated throughout the entire system for what may seem at 
first to be a simple coie modification requirement. 
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c. 



VARIABLES AFFECTING MAINTENANCE COSTS 



As mentioned in the preceding section, an accurate esti- 
mate of maintenance costs for a particular program is very 
difficult. Sommervile [Ref. 25 : pp. 198 -199] has identi- 
fied five relatively unpredictable factors that contribute 
to the difficulties involved in deriving cost estimates for 
maintenance. These factors include: 



( 1 ) The application be ing s upp orted. The better the 

application Being supporte3 _ 5y software is under- 
stood, the better the system requirements can be 
stated. The more definitive the system requirements 
are, the less perfective maintenance will be 

reguired in the future. 

(2) Lifet ime of the pr ogr am. Toward the end of the 

program life, a aefer loration of the program struc- 
ture occurs due to the multitude of modifications 
that the typical program has gone through. 

Historical evidence suggests that program lifetime 
is traditionally much longer than originally esti- 
mated. Many systems today are still running on 
programs that were coded in the early 1960’s. 

(3) Depen den ce of the program on its external en vi ron- 
me n t. The closer a program is tiecl "To factors in 
TEe external environment, the more flexible and 
expandable that program must be to accommodate modi- 
fications due to changes in environment in which it 
operates . 

(4) Staff stability. It is normally easier for the 

original - author of the program to make changes than 
for another programmer who must gain an under- 
standing of the program by studying its documenta- 
tion. Pressman [Ref. 1] uses the term alien code 1 ' 
to describe these programs that are extremely diffi- 
cult to understand by those that must maintain them. 
Reasons for alien code include: (1) no current 

member on the maintenance staff was involved in the 
development of the program, (2) poor design and 
documentation of the program, and (3) modularity and 
structure design concepts were not used in the 
development of the program. High turnover within the 
programming profession has made it a rare occurrence 
for the same individuals to develop and maintain a 
pregram throughout its life cycle. 

(5) Hardw are stabi lity ♦ Software is designed to be 
ccmpafTBle wiTK The hardware that will support it. 
Changes, to the hardware configuration will likely 
result in requirements for software modification. 
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D. 



INTANGIBLE MAINTENANCE COSTS 



The direct cost of maintenance, although considerable, 
may be of secondary concern when compared with the less 
obvious and intangible cost of maintenance. A quote by 
Daniel McCraken [Ref. 1] summarizes one of these intangible 
costs, the development opportunities that are lost due to 
the resources that must be allocated to maintenance efforts: 

"Backlogs of new applications and major changes that 
measure in years are getting longer. As an industry, we 
can't keep up — let alone catch up — with what our users 
want us to do. " 

McCraken alludes to what Pressman 1 call a "maintenance- 
bound" software development organization which is no longer 
capable of producing new software because all its resources 
are devoted to the maintenance of existing software. 
Pressman lists other intangible costs including: 

— Customer dis sati s factio n due to the untimely response 
By BBe soBBware development organization to the user's 
development and maintenance reguests 

— Se du ction of soft ware (quality brought about by latent 
errors introduced during 'the maintenance of software 

— Uyheaval of de velopment efforts as personnel and other 
resources are TT puT±ed TT— to work - on maintenance tasks 



E. B BIDDING MAINTAINABLE SOFTBARE 

Economic and efficient support of software is best 
achieved when its maintainability is integrated into the 
total development effort from day one. The maintainability 
of software can be quantitatively measured based on the ease 
by which it can be understood and changed [Ref. 26 : p. 14]. 

Software under standability is a function of the design 
and documentation. It is easy to understand due to its 
logical and simple structures, and it is supported with 
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documentation that permits an examination of the implementa- 
tion without losing an understanding for the entire picture. 
Software changeability, on the other hand, is a function of 
the design and implementation. As an example, implementa- 
tion of modular independence facilitates changes to a 
selected segment by minimizing the degenerative affect on 
other segments. 

% 

1 . Str uctu red M ethodol ogy 

Building maintainable software is based on the usage 
of a set of software engineering tools and techniques that 
together form a structured methodology for software develop- 
ment. As the authors [Sef. 26] write, the principal elements 
of this structured methodology include: 

11 Structured Analysis. A process for developing the 
functional^" 3afa, and interface requirements of the 
software design by constructing a logical model of the 
system process. 

Structured D esign . The process of subdividing the soft- 
ware Info Kierarchial modules in a way that tends to 
minimize module independence. 

Struc tur ed Pr ogra m min g. The discipline of implementing 
file control "structure of software modules using a 
restrictive set of structures. 

Pr og ram D esi gn L an guage (PDL) . Language processors that 
are used to document software designs in a structured 
top-down manner." 

Although there are many other disciplines and concepts that 
must also be considered as part of a complete structured 
methodology, such as top-down implementation, structured 
walk-throughs, chief programmers teams, and so forth, these 
managerial disciplines are used primarily for the software 
development effort. The methodologies underlined above will 
have a visible affect on the software product long after its 
development is completed. A detailed description of the 
characteristics of each of these elements i^ provided in 
Appendix C. 
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Although there are many other disciplines and 
concepts that must also be considered as part of a complete 
structured methodology, such as top-down implementation, 
structured walkthroughs, chief programmers teams, and so 
forth, these managerial disciplines are used primarily for 
the software development effort. The methodologies under- 
lined above will have a visible affect on the software 
product long af'ter its development is completed. A detailed 
description of the characteristics, as provided in [Eef. 26 ] 
of each of these elements is provided in Appendix B. A mere 
general discussion of each will now be presented. 

2 • Struct u red Analysis 

Structured analysis is often considered the starting 
point in the set of structured design techniques. The main 
objective of structured analysis is to build a logical model 
of the desired system. This should be done to the greatest 
extent possible without premature consideration of physical 
implementation . 

In its simplest form, the logical model is a picto- 
rial representation with accompanying narration describing 
the functions, and their interrelationships, of the system 
that they comprise. Examples of the some of the most popular 
forms of these graphical representations include process and 
information flowcharts, data flow diagrams, hierarchy chart 
plus input- processing-output chart (HIPO charts), and 
procedure analysis charts. 

The net result of the this structured analysis 
process should be a logical model that defines the complete 
system which reflects all facets of the system specification 
and software requirement document. The model should be a 
form of communication easily understood by both technical 
and nontechnical personnel, alike. Through the use of this 
model, the system analyst should be to develop system 
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requirements without undue consideration of physical imple- 
mentation constraints. On the other hand, nontechnical users 
should readily be able understand how the required functions 
fit together within the context of the whole system. 

3. Structured De sign 



Structured design is the process of of decomposing 



the software design into hierarchial modules in* a manner 
that leads toward independence of modules. Benefits of 
structured design to the development and maintenance of 
software include increased understandability of the system 
and a minimization of the cost inherent in modification. 



Modularity is the key element of structured design. 
It allows for software to be better managed. Large mono- 
lithic (i.e., single module) programs are often unintellige- 
able to the reader. Modularity is based on a "divide and 
conquer" concept, breaking complex problems into comprehen- 
sible and manageable components. Two primary measurements 
of modularity are (1) cohesion, and (2) coupling. 



-- Cohesion is the "relative functional strength" 
fSel. T : p. 158] of a module. A module is said to be 
cohesive if it performs a single task within a 
program, requiring little interaction with other 

S rogram code external to its boundaries. In general, 
esign should attempt to realize the highest degree 
cf module cohesion. 

-- Cou pling is a measurement of the connectivity among 
other modules. It is based on (1) the interface 
complexity between modules, (2) the place at which 
entry are reference are made to a module, and (3) the 
type of data that passes across the interface TRef. 1 
; pp. 161 - 162 1. The designer should strive for the 
lowest degree of module coupling. 

Clearly, then, the objective of structured design is 

to minimize the relationships between modules through the 

maximization of the functional strength of each. 
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4. 



St r uct ured P rogramm i ng 

Structured programming is the discipline of imple- 
menting module functionality through the use of a limited 
set of programming structures. 

Structure programming uses top-down design by 
starting with the top-level module and decomposing it into 
lower-level modules that it will call upon. This decomposi- 
tion process repeated as often as necessary until the 
bottom- level modules are defined. At this bottom level, 
modules make use of built in operators and functions; they 
do not call on any other module. Each module is separately 
coded using the basic set of program instructions. An objec- 
tive of structured programming is to make the design match 
the structure of the program. 

Any program, regardless of its size and complexity, 
can be designed using three basic programming structures. 
The use of this set of programming structures reduces the 
procedural design of the program to a small number of 
predictable operations, greatly facilitating the development 
and maintenance of software. These structures are illus- 
trated in Figure 4.3. 

5 . P rogra m Design Langu age 

Program Design Languages (PDL) are language (text) proces- 
sors that are used to document software designs in a struc- 
tured top- down fashion. The goal of a PDL is to replace or 
support traditional forms of documentation of program 
desi gn. 

The primary benefits of a PDL are: (1) the documen- 

tation that it produces is normally easier to read and 
understand than flow charts, and (2) the documentation is 
always easier to change than are flow charts. Both of these 
advantages are essential during maintenance activities. 
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F 



PASTING COMMENTS 



The maintainability cf software is inseparable from the 
degree of quality that was built in prior to the maintenance 
phase. Sound software engineering practices, coupled with 
the implementation of managerial controls in maintenance 
activities, offer the key to improved productivity and the 
reduction costs associated with maintenance .acti vi ties. 
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VI. DOD STANDARDIZATION AND SPECIFICATIONS 

A. INTRODUCTION 

In 1980, it was estimated that DoD spends about $7 
billion a year on software [Ref. 28. : p. 3]. This amount 
has been steadily increasing as DoD becomes increasingly 
dependent on larger and more complex software products to 
support this generation of sophisticated weapon systems. The 
upward-spiralling trend in the cost of DoD software has 
naturally become an area of great concern to officials in 
both military and government. This concern has led to a 
number of management initiatives in DoD, several of which 
will he discussed in this chapter. At the heart of these 
initiatives is the standardization of computer technology 
and software. Standardization is seen a means for reducing 
costs associated with the development, operations, and 
support of DoD computer systems. 

B. SPECIFIC INITIATIVES 

In her article [Ref. 29 : pp. 37 - 47] Becker describes 
three distinct, but interrelated, initiatives that reflect 
the DoD standardization effort. These initiatives are as 
follows : 

(1) The Army's development of a Military Computer Family 
(MCF) 

(2) The adoption of Ada as a higher order language (HCL) 
for development of embedded computer software. 



6 An instruction set architecture can be described as the 
rules and procedures by which hardware executes instructions 
or computer software. It can also be defined as the struc- 
ture of a computer that a programmer must know to write 
time-independent machine language [Ref. 29 : p. 39]. 
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